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Abstract 

The No Child Left Behind Act of 2002 has increased emphasis on “closing 
the gap” between the achievement of African-American and 
disadvantaged children and that of their respective peers. Using the 2001 
results of Kentucky’s accountability tests (e.g., CTBS-5, KCCT), Chi- 
square analyses were performed to determine whether, when 
disaggregated by quartile (CTBS-5) or proficiency-level (KCCT), a 
significant difference existed for each school between the distributions of 
disadvantaged and minority students and those of their respective peers. 
Chi-square analyses were additionally performed to determine whether 
statistically significant relationships existed between exogenous school 
characteristics (e.g., school size, school poverty level) and the equity of 
student achievement. Results of the study indicated that a majority of 
elementary, middle, and high schools displayed inequitable student 
outcomes between disadvantaged and minority students and their 
respective peers on both assessments. Moreover, school size was shown to 
relate significantly to the equity of student assessment results at the 
elementary and high school levels. 

Prior to No Child Left Behind, only 17 states had required that 
student achievement data be disaggregated (“Quality Counts”, 2002). 
With equity now a component of Adequate Yearly Progress (AYP)(Center 
on Educational Policy, 2002), it has become critical the current level of 
equity of achievement between African-American and disadvantaged 
students and their respective peers be established so that progress may be 
measured. The purpose of this study was to describe the degree to which 
ethnic and socioeconomic equity existed in Kentucky elementary, middle 
and high schools in 2001, and to assess whether a trend exists between 
equitable student outcomes and overall school socioeconomic -level and 
school size. 
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Nature of Public School Accountability 

In general, public school accountability models have been 
constructed around a framework of systemic goals, standards of school 
performance, a means of school performance measurement, and a system 
rewards and sanctions assigned to schools based upon varying levels of 
school performance (Hanushek & Raymond, 2001). Public school 
accountability systems have sought to leverage change by opening schools 
to public scrutiny for the purpose of placing pressure upon schools to take 
steps to increase student test scores (Gullatt & Ritter, 2000; Ladd, 2001). 
Prior to the passage of the No Child Left Behind Act (NCLB Act) of 2002, 
48 states and the District of Columbia had enacted accountability systems 
that utilized education goals and standards in core subject areas to raise 
aggregate student achievement (“Quality Counts”, 2002). 

Accountability in Kentucky 

In 1992, Kentucky’s accountability system, at that time the 
Kentucky Instructional Results Information System (KIRIS), was 
implemented as part of the Kentucky Education Reform Act (KERA). 
KIRIS was designed to reflect the degree to which schools were 
improving the effectiveness of their instructional program within the 
context of the Kentucky’s learning goals, as measured by both cognitive 
and non-cognitive indicators. A system of rewards and sanctions based 
upon schools’ progress toward “improvement goals” was implemented as 
an incentive for schools to progress toward KIRIS goals (KDE, 1998). 

With KIRIS, each individual school was assigned a KIRIS 
improvement goal every two-years based upon the difference between 
each school’s current KIRIS accountability index and the 20-year 
statewide goal of 100. This method of establishing improvement goals 
was intended to allow interim goals to be individualized for each school, 
with initially low-performing schools having to show larger biennial gains 
than schools that were initially high-performing (KDE, 1998). 

Schools meeting or exceeding their improvement goal were 
designated as being in “Rewards”, which entitled them to receive 
monetary rewards from the state. Schools failing to meet their 
improvement goal, but that scored at or above their baseline, were 
identified as “Maintaining”. These schools were not eligible for rewards 
and were required to submit a school improvement plan. Schools whose 
KIRIS scores dropped below their baseline were designated as being in 
“Decline”. Depending upon the degree to which the school was in 
Decline, consequences ranging from a required school improvement plan, 
the assignment of a “highly skilled educator”, and the option of parents to 
transfer students to another school were all possible (KDE, 1998). 
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From KIRIS to CATS. Highly criticized for several reasons, including the 
lack of a national norm-referenced test as part of each school’s 
accountability index, KIRIS was replaced by the Commonwealth 
Accountability Testing System (CATS) in 1998. CATS differed from 
KIRIS in several ways, but most fundamentally in the addition of the 
Comprehensive Test of Basic Skills Survey Edition (CTBS-5) in three 
grade levels (3, 6, 9) to compose five percent of each school’s CATS 
index. The remaining 90% to 95% of the CATS index for elementary, 
middle, and high schools was composed of the results of the Kentucky 
Core Content Test (KCCT), writing portfolios, and non-cognitive 
indicators (KDE, 2000). 

Provisions of NCLB 

The NCLB Act of 2002 established federal public school 
accountability guidelines requiring consequences for schools failing to 
meet accountability goals in terms of both aggregate and disaggregate 
student achievement, primarily in the areas of reading and mathematics. 
The NCLB Act has stipulated that students must be tested in reading and 
mathematics each year in grades 3 through 8, and at least once in grades 
10 through 12, with science testing to be required later in selected grades. 
Schools will be held accountable for overall student achievement, as well 
as the equity of disaggregated student outcomes, inclusive of major racial 
and economic categories (CEP, 2002). 

The consequences for failing to meet aggregate and disaggregate 
accountability goals increased with each year of non-compliance. 
Specifically, schools failing to meet their performance goals for two 
consecutive years must receive technical assistance from the district, with 
students having the option to transfer to another school within the district. 
Schools failing to meet their goals for a third consecutive year would 
continue to receive technical assistance, must allow intra-district school 
choice, and allow eligible students to use their portion of Title I funds to 
purchase tutoring or other services directly from the district or an outside 
agency. A fourth consecutive year of failure would continue the previous 
consequences, but also impose re-staffing or other fundamental changes. 
After a fifth consecutive year of failure, a change in school governance 
was mandated (i.e., charter school, privatization of management services, 
state takeover of operations) (CEP, 2002). 
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Impact of Exogenous School Characteristics on Equity 

The emphasis of the NCLB Act on narrowing the achievement 
gap between minority and disadvantaged students and their peers, and the 
subsequent mandatory disaggregation of student assessment data as part of 
the accountability system (CEP, 2002), has constituted a major shift in 
focus for most state accountability systems. For example, only 17 states 
even required disaggregated student testing results to even be reported in 
2001 (“Quality Counts”, 2002). 

Additionally, the identification or development of valid and 
reliable measures of student performance for accountability purposes has 
generated significant controversy (Ladd, 2001), with several studies 
having shown that exogenous factors have influenced aggregate student 
achievement on accountability tests. For example, approximately 52% to 
62% of the variance in aggregate school or school district performance on 
accountability assessments has recently been shown to vary with student 
socioeconomic factors in both Illinois and Ohio (Lyons, 2001; Sutton & 
Soderstrom, 1999; Wilson & Martin, 2001). Moreover, it has been 
asserted that exogenous school characteristics such as school size, and 
overall poverty level have impacted the gap between the achievements of 
students from poverty as compared to their peers (Johnson, Howley, & 
Howley, 2002; “Low-income children”, 2001). 

As Kentucky adapts CATS to comply with the NCLB Act by 
developing, among other things, a definition and means of measuring the 
equity of student outcomes on existing accountability assessments, it 
stands to reason that the current state of equity should be established so 
that progress can be fairly and accurately gauged. Additionally, the 
impact of exogenous variables on equity would possibly have implications 
as policy-makers look to adapt Kentucky’s existing accountability system 
to meet federal mandates. Therefore, the purpose of this study was as 
follows: 

(1) To establish the an equity baseline for Kentucky’s public 
elementary, middle, and high schools on the reading and mathematics 
subtests of the state’s accountability assessments prior to the 
implementation of the tenets of NCLB. 

(2) To investigate possible relationships between exogenous 
characteristics (e.g., school size, school poverty level) of Kentucky’s 
public elementary, middle, and high schools on the equitability of student 
outcomes on the math and reading subtests of the state’s accountability 
assessments 
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Methodology 

Participants 

The sample consisted of 729 of 782 elementary schools, 164 of 
213 middle schools, and 220 of 245 high schools from the State of 
Kentucky. Schools were eliminated that did not exhibit the grade level 
structure of P-5, 6-8, and 9-12. 


Instruments 

The 2001 results of the math and reading subtests of the 
Comprehensive Test of Basic Skills version 5 (CTBS-5) and the Kentucky 
Core Content Test (KCCT) were used to assess student achievement for 
elementary, middle, and high schools in Kentucky. These results were 
obtained in a disaggregated format from the Kentucky Department of 
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Education (“Spring 2001 Kentucky Performance Report - National Norm 
Referenced Test (NRT) Data Disaggregation”; “Spring 2001 Kentucky 
Performance Report - Data Disaggregation Pages with Scaled Scores”). 
The subtests were not administered at every grade level. Table 1 
summarized the grade-levels for which the reading and mathematics 
subtests of the CTBS-5 and KCCT assessments were administered. 

CTBS. The CTBS was a norm-referenced, multiple-choice test of 
student basic skills in 11 areas for kindergarten through Grade 12. The 
validity and reliability of the CTBS has been considered good. Version 
four, which was replaced by version five in 1990, was characterized as an 
exemplary general achievement test battery (Hopkins, 2002). 

KCCT. The KCCT was a criterion-referenced test designed to 
assess students against the education standards of the State of Kentucky. 
The KCCT assessed students over seven content areas, inclusive of 
reading and mathematics, using a combination of multiple-choice and 
open-response questions (KDE, 2001). There was no information 
available regarding the validity or reliability of the KCCT. 
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SCHOOL ENROLLMENT 


FREE/REDUCED LUNCH 
PARTICIPATION 

ELEMENTARY SCHOOL 




1 

69 to 254 


1% to 40.00% 

2 

257 to 368 


40.17% to 52.13% 

3 

370 to 474 


52.26% to 64.45% 

4 

475 to 577 


64.60% to 77.35% 

5 

578 to 1292 


7766% to 97.86% 

MDOLE SCHOOL 




1 

200 to 428 


3.74% to 32.85% 

2 

438 to 552 


33.12% to 41.79% 

3 

554 to 675 


42.20% to 52.09% 

4 

690 to 810 


52.30% to 63.12% 

5 

812to 1552 


63.34% to 88.77% 

HIGH SCHOOL 




1 

83 to 417 


2.14% to 2564% 

2 

420 to 615 


26.14% to 35.00% 

3 

618 to 813 


35.09% to 45.12% 

4 

827 to 1074 


45.78% to 5860% 

5 

1093 to 2012 


5864% to 92.96% 


Procedure 

Federal Free/Reduced Lunch Data for all public elementary, 
middle, and high schools in the State of Kentucky was obtained from the 
Kentucky Department of Education (“October free and reduced price 
data — 2001-2002”). This list was compared to the school-level reports for 
the CTBS-5 and KCCT assessments for 2001, with schools not appearing 
on all three lists eliminated from the samples. 

Size and socioeconomic classification. After the samples of 
elementary, middle, and high schools had been established, the school 
enrollment and percent free/reduced lunch participation data was used 
assigned schools to a quintile based upon school size and then a quintile 
based upon school socioeconomic level. Table 2 identifies the cut points 
for the socioeconomic and school size quintiles for elementary, middle, 
and high school students. 

School-level disaggregated assessment data was obtained from 
the Kentucky Department of Education for the CTBS-5 and KCCT 
assessments. Disaggregations for free/reduced lunch participation (e.g., 
participants, non-participants) and ethnicity (e.g., Caucasian, African- 
American) were matched with each elementary, middle, and high school 
in the three samples. For each subtest, the disaggregated data reported the 
number of students from each school that scored in each national quartile 
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for the CTBS-5, and the number of students classified as Novice, 
Apprentice, and Proficient/Distinguished for the KCCT. 

Equity Analysis. A Two-Way Chi-square was used to determine 
whether a significant difference (p < .05) existed between the distribution 
of students across either the CTBS-5 quartiles, or the KCCT 
classifications, for students disaggregated based upon ethnicity or 
socioeconomic level. Elementary, middle, and high schools were 
classified as equitable or inequitable based upon the Chi-square analysis 
for the reading and mathematics subtests for each disaggregation. The 
number of elementary, middle, and high schools classified as equitable or 
inequitable were reported. 

School size and socioeconomic level. The degree to which school 
size or school socioeconomic level impacted the equity of student 
achievement was investigated using a Two-way Chi-square analysis. The 
Chi-square was used to determine whether a significant relationship (p < 
.05) existed between the frequency with which schools were designated as 
equitable or inequitable and the frequency with which they were 
designated in the either the school size quintiles or the socioeconomic 
quintiles. If a significant relationship existed, the frequency distributions 
were examined to determine whether the Chi-square was the result of a 
discernable trend or whether the statistical result appeared spurious. 

Results 


ELEMENTART MIDOtE HIGH 

N=707 N= 165 N=220 


ECONOMIC STATUS READING MATH READING MATH READING MATH 


EQUITY 

287 

236 

35 

41 

77 

79 


40.6% 

33^4% 

21.2% 

24.8% 

35.0% 

35.9 

INEQUITY 

156 

117 

88 

95 

122 

114 


211% 

16.5% 

53.3% 

57.6% 

55.5% 

51.8% 

EQUITY 

2 

3 

15 

16 

32 

14 


0.3% 

0*% 

9.1% 

9.7% 

14.5% 

6.4% 

INEQUITY 

8 

0 

48 

43 

39 

41 


1.1% 

0% 

29.1% 

26b1% 

17.7% 

18^% 
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Chi-square analyses were performed for elementary, middle. 



ELEMENTARY 

N=707 


MIDDLE 

N=165 


HIGH 

N=220 

ECONOMIC STATUS 

READING 

MATH 


READING 

MATH 


READING 

MATH 

EQUITY 

309 

329 


34 

22 


65 

57 


43^% 

46.5% 


20.6% 

13.3% 


29.5% 

25.9% 

INEQUITY 

189 

167 


126 

138 


141 

150 


26.7% 

236% 


76.4% 

83.6% 


64.1% 

68.2% 

ETHNICITY 









EQUITY 

106 

97 


18 

18 


25 

24 


15.0% 

13^% 


10.9% 

10.9% 


1 1.4% 

10.9% 

INEQUITY 

63 

72 


50 

50 


48 

49 


8.9% 

10.2% 


30.3% 

30.3% 


21.8% 

22.3% 


and high schools to determine whether student achievement on the 
reading and mathematics subtests of the CTBS-5 and KCCT was 
equitable between disadvantaged students and their peers and African- 
American students and Caucasian students. If less than 10 students 
were present in an achievement category, the disaggregated test results 
were not compiled by the KDE due to student confidentiality reasons. 
As a consequence, a large number of schools could not be classified as 
equitable or inequitable in some analyses. Tables 3 and 4 summarized 
the equity classifications for elementary, middle, and high schools. 

Chi-square analyses were performed to investigate whether 
statistically significant and discernable trends existed between the 


equitability classification of schools, and the size and economic 


ELEMENTARY 


MIDDLE 


HIGH 

CTBS-5 

READING MATH 


READING MATH 


READING MATH 

FRL 

12.368* 3.353 


3.353 11.236* 


7.644 5.252 

ETHNICITY 

32.456“ 27.501 ** 


3.569 5^43 


9.092 3.152 

KCCT 






FRL 

5.571 6.523 


0.403 1.571 


6.070 3622 

ETHNICITY 

9.000 


2.509 8.542 


1.725 9.113 


“ .^significant at the p - .05 level 
" X 1 significant at the p = .001 level 

classifications of schools. Chi-square values for these analyses were 
summarized in Tables 5 and 6. Significant Chi-square values have been 
identified, with significant, but indiscernible trends being designated as 
such. 
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ELEMENTARY 


MIDDLE 


HIGH 

CTBS-5 

READING 

MATH 


READING 

MATH 


READING 

MATH 


FRL 

21X116** 

18.515** 


7.303 

5.796 


46.462'* 

39.610“ 


ETHNICITY 

8.802 

1 8.769++ 


8.059 

6.946 


17.676** 

9.954+ 

KCCT 










FRL 

1.816 

5.805 


5.276 

4.706 


23.239** 

17.285** 


ETHNICITY 

.900 



5.994 

8.558 


4.103 

10.198* 


* X 7 significant at the p = .05 level 
** X 7 significant at the p = .001 level 
+ X 7 significant at the p = X)5 level, indiscernible trend 
+ X 7 significant at the p = .001 level, indiscernible trend 


Discussion 

This study was designed strictly as a descriptive investigation of 
the state of equity in Kentucky, and of potential trends that might exist 
between schools of different sizes and of different poverty levels. There 
are factors (e.g., teacher quality, principal attributes and effectiveness, 
class size, etc..), which certainly limit the generalizability of this study. 
Moreover, this study addresses only the equitability of student scores, 
rather than both equitability and excellence of student scores. 

Despite the limitations of this descriptive investigation, it is asserted that 
the value lies in the following areas: (a) the heightening of awareness of 
the current status of equity of student assessment outcomes in Kentucky, 
(b) the heightening of the awareness of the potential impact of exogenous 
variables on the equity of student performance and the policy implications 
relative to adaptations of CATS for NCLB. 

The State of Equity 

Socioeconomic and ethnic equity analyses relative to student performance 
on the reading and mathematics subtests of the CTBS-5 and KCCT for 
Kentucky’s elementary, middle, and high schools indicate that in no case 
were a majority of schools shown to be equitable. In general, it appeared 
that elementary schools faired better than middle and high schools in terms 
of equity, although the number of elementary schools with insufficient 
economic and racial diversity to warrant data disaggregation left questions 
unanswered concerning the equity of elementary schools, particularly in 
terms of ethnicity. 
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Overall, schools displayed less inequity on the KCCT than the 
CTBS-5 in terms of achievement in the areas of mathematics and reading. 
The fact that Kentucky public school curriculum is aligned with the same 
standards for which the KCCT was written might explain this difference, 
with the focus of school-wide curriculum on the core content negating 
potential inequity that surfaced on the generic basic skills test. 
Conversely, it is postulated that public schools that have failed to align 
their curriculum to the Kentucky Core Content for whatever reason (e.g., 
technical inaptitude, lack of leadership) may have displayed more inequity 
on the KCCT than those that did. 

Impact of Exogenous Variables 

So far as the impact of exogenous variables on the equity of 
student achievement, it was noted that for elementary and high schools, 
there were more significant trends in terms of school size, than overall 
socioecononomic status. 

School size. A significant trend for larger schools to display 
more inequity in terms of disadvantaged students as compared to their 
peers existed for both elementary and high schools on the reading and 
mathematics subtests of the CTBS-5. This pattern existed only for high 
schools on the KCCT. Although there was not a pattern across subtests, 
school size did result in significant trends on the CTBS-5 reading subtest 
and the mathematics subtest of the KCCT. 

The results of the analyses is congruent with research that asserts 
that large schools benefit advantaged students at the expense of 
disadvantaged students, at least for the elementary and high schools. 
Although grade level structure was controlled for, it is quite possible that 
other differences exist between schools for which there were no control. 
The fact that middle schools did not display the same pattern as did 
elementary and high schools further reinforces this conclusion. 

School size factors did not impact the equity between African- 
American students and their peers in as consistent a fashion as with the 
school size factors. A closer examination of the schools included in the 
sample may very well show additional confounding factors that are 
concurrent with school size, therefore impacting the Chi-square results. 

School socioeconomic level. School socioeconomic level had no 
discernable impact on high school equity, and appeared to impact middle 
schools only in terms of the equity between disadvantaged students and 
their peers on the math subtest of the CTBS-5, but with no discernable 
trend. The inequity of elementary student scores in terms of ethnicity on 
the reading and mathematics subtests of the CTBS-5, and in terms of free 
and reduced lunch participation on the reading subtest of the CTBS-5 were 
Vol 27, No. 3, Mar 2004 
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the only significant result of this analysis. However, there was not a clear, 
discernable trend evident. 

Policy Implications 

The establishment of education state-level accountability systems 
over the past several years has led to as comprehensive an understanding 
as ever of the aggregate performance of schools on state-mandated 
accountability assessments. However, the NCLB Act of 2002 has made it 
critical that education policy makers, as well as public school educators, 
have systematic means in place of gauging the performance of 
disadvantaged and minority students on these same tests when compared 
to their peers. 

Kentucky’s accountability system, which has been highly 
regarded nationally (“Quality Counts, 2002”), has likewise provided 
education policy-makers, the general public, and public school educators 
with a comprehensive understanding of how schools have performed 
relative to Kentucky’s educational goals. Until recently, however, the 
focus of Kentucky’s accountability system has not been on disaggregated 
student results, and even now the general public is not fully aware of how 
schools are performing in terms of equity. 

The 2001 disaggregated accountability assessment results 
indicate that a majority of Kentucky schools had inequitable outcomes, 
particularly in the upper grades. Additionally, it appears that in terms of 
equity, large schools are not as successful as smaller schools. This trend 
warrants further study, particularly in a climate of efficiency where school 
districts are encouraged to consolidate resources, inclusive of school 
building administration. 

Certainly, more investigation is needed to support any assertions that a 
school’s size, economic status or both relate causally to equity on student 
assessments. Moreover, equity in the absence of excellence will not bode 
well under any accountability system. Therefore, it seems that a research 
agenda for future equity studies should target the level of aggregate 
student assessment results along with the disaggregated results. 
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